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1 Introduction 

The fundamental problem underlying all multi-criteria decision analysis (MCDA) 
problems is that of dominance between any two alternatives: ‘Given two alter¬ 
natives A and B, each described by a set criteria, is A preferred to B with respect 
to a set of decision maker (DM) preferences over the criteria?’. Depending on 
the application in which MCDA is performed, the alternatives may represent 
strategies and policies for business, potential locations for setting up new facil¬ 
ities, designs of buildings, etc. The general objective of MCDA is to enable the 
DM to order all alternatives in order of the stated preferences, and choose the 
ones that are best, i.e., optimal with respect to the preferences over the criteria. 

The preferences over the criteria typically represent the relative importance 
of the various criteria to the DM, which is typically expressed either in the 
form of an ordering (e.g., a ranking) over the criteria, or if available, a set 
of numeric weights on each criterion. Most of the existing quantitative multi¬ 
criteria decision methods (QnMCDMs) [TOl [71] assume that it is possible to (a) 
completely rank the criteria in order of their importance, and/or (b) precisely 
quantify the degree of importance of criteria (e.g., as weights). Some formalisms 
such as Analytic Hierarchy Process (AHP) [75lll], SMART [7], ELECTRE (6] 
[H SSI, TOPSIS [21 HZ], VIKOR [Sg and PROMETHEE [H [HIS] elicit 
relative importance of criteria using a matrix that specifies the pairwise ordering 
of the criteria, which is then used to generate weights for each criteria by a 
mathematical transformation ng. In formal terms, this amounts to assuming 
that the preferences will always be a weak order IH- The weak ordering is then 
typically transformed into a quantitative (numeric) scale, and the transformed 
importance values are then used to evaluate and compare candidates. 

However, it is not surprising that generation of weights based on qualitative 
assessments of pairwise relative importance of criteria often leads to instabil- 
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ity in rankings of alternatives and their high sensitivity to changing weights 
or addition of alternatives [14]. In addition, DMs may not understand the as¬ 
sumptions involved in generating the weights. Moreover, in many applications, 
the DM’s preferences may be naturally incomplete and/or imprecise, especially 
when the stakes are really high (e.g., trade-offs in facility location for setting 
up new nuclear reactors). For example, a building designer as a DM may state 
that increased energy efficiency and pollution efficiency (lesser pollution) are 
more important than cost efficiency when evaluating candidate designs, with¬ 
out quantifying the relative importance between cost efficiency and either of the 
other criteria (imprecision), or without even specifying any relative importance 
between energy efficiency and pollution (incompleteness). A reason for such 
imprecision may be that precise weights for each criteria may not be available; 
another reason may be that the DM does not want to commit to precise weights 
a priori to avoid sensitivity of rankings to initial weight estimation errors. 

In this article, we present and summarize a recently developed MCDA frame¬ 
work that orders the set of alternatives when the relative importance preferences 
are incomplete, imprecise, or qualitative in nature. To this end, we allow the DM 
to specify the preference for a decision problem with a set A = {Ai, A 2 , ...A^} 
of criteria as a strict partial order over A, i.e., the relative importance prefer¬ 
ence \> is represented by a binary relation [> C A x A. It is easy to see that such 
a criteria set can be readily represented as a directed acyclic graph. We discuss 
a dominance relation recently developed in artificial intelligence literature 
P5] . which has the desirable property that it is a partial order whenever the 
relative importance preference > is an interval order relatior0. The ordering 
of alternatives based on is provably correct, i.e., if A )~d B, then it can 
be argued using principles of rational choice that A is indeed preferred to B. In 
that sense, the partial ordering produced by can be considered as a reference 
ordering that all other rankings must be consistent with. We discuss methods 
of comparing the orderings produced by >-d and other MCDA ranking methods, 
drawing from recent advances in order theory. 


2 Background 

The following is the typical decision process for multi-criteria decision problems 
such as the above. 

1. First, the stakeholder defines a set of attributes or criteria on which the 
alternatives can be evaluated and compared. These criteria can be either 
quantitatively or qualitatively evaluated, e.g., the noise level during con¬ 
struction of a pavement type could be rated on a relative, ordinal scale 
(Low/Medium/High), or in terms of the maximum decibel level antici¬ 
pated during construction. 

^For > to be an interval order, it must be irreflexive, transitive, and for all Xi, Xj, Xi^, Xi S 
X, Xi t> Xj and Xj, > Xi must imply that either Xi t> Xi or > Xj mm 
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2. Next, the stakeholder uses his knowledge and experience to evaluate each 
alternative with respect to each of the attributes on a qualitative scale 
(e.g., ranking, partial order) or quantitative scale (e.g., utility function), 
such that the evaluation of two designs with respect to a criterion indicates 
their relative desirability. Further, the stakeholder may specify relative 
importance preferences over the criteria quantitatively in terms of numeric 
compensation (e.g., weights) for the criteria, or qualitatively (e.g., energy 
consumption is more important than noise levels). 

3. Finally, an appropriate multi-criteria decision method m is chosen, which 
takes into account the evaluations of the alternatives with respect to the 
criteria, as well as relative importance preference over the criteria and 
produces an ordering of the alternatives. Each decision method differs 
primarily in the set of rules of rational choice that it uses to compare 
alternatives, and hence the orderings they produce may accordingly differ. 

Note that three factors are crucial to the validity and accuracy of the deci¬ 
sion: a) the choice of whether each of the criteria will be evaluated on a qual¬ 
itative or quantitative scale; b) the choice of whether the relative importance 
preference will be qualitatively or quantitatively specified; and c) the choice of 
the decision method which determines how the alternatives will be compared 
with respect to each other based on the stated preferences. 

2.1 Need for a Qualitative Approach to MCDM 

Quantitative MCDMs, i.e., QnMCDMs such as multi-attribute utility theory 
(MAUXjl |74j require the criteria evaluation scale and the relative importance 
preferences to be all quantitative. Some other MCDMs such as AHP m, 
SMART, ELECTRE, TOPSIS, and other outranking methods (see [TT]) that fo¬ 
cus on certainty allow the stakeholder to specify qualitative input, but eventually 
convert them into quantitative scores according to some rules of assignment. All 
the above QnMCDMs apply quantitative decision rules to evaluate dominance, 
i.e., process (quantitatively elicited or qualitatively elicited and quantified) crite¬ 
ria evaluations and preferences in order to produce a ranking of the alternatives. 
However, in many design environments there is incompleteness and imprecision 
in the stakeholder’s evaluations and preferences over alternatives. 

2.2 Limitations of QnMCDMs 

The challenge in using QnMCDMs for decision making in design problems is 
that when three or more alternatives with multiple attributes are involved, the 
commonly employed techniques for normalizing evaluations, weighting the cri¬ 
teria and multi-attribute ranking cannot guarantee the selection of the best 

■^The notion of utility automatically includes a notion of uncertainty which is quantified in 
the form of probabilities. In this article, we do not consider settings where there is uncertainty 
over the outcomes of the alternative choices. 
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alternative in an unambiguous, rational, and consistent manner. While Qn- 
MCDMs have been largely successful when alternatives can be quantitatively 
evaluated with respect to criteria and weights can be precisely estimated, they 
are inappropriate for problems where some preferences are naturally qualitative 
as shown below. 

1. Incomplete Preferences: Stakeholders may assert that two alternatives 
are incomparable on the same criterion. Similarly, stakeholders may not 
always be able to totally rank the criteria [5?1[5]. However QnMCDMs as¬ 
sign numeric weights to criteria, assuming that there is a total or weak or¬ 
dering of the criteria. To deal with incompatibilities, different QnMCDMs 
extrapolate the stakeholder’s preferences and ‘fill in the blanks’ based on 
certain assumptions to obtain a complete ranking. This questions the va¬ 
lidity of use and the accuracy of the results produced by QnMCDMs for 
the problem. 

2. Semantic Discrepancies: Numeric weights often have no meaning more 
than the relative ordering of criteria and the precise meaning of criteria 
weights is itself often not well understood by stakeholders [3 [H]. In 
particular, there is a disconnect between the intuitive meaning of the 
values in the scale as understood by the stakeholder versus the algorithmic 
meaning of the values in the same scale that is assumed by the QnMCDM 
while comparing alternatives m- 

3. Imprecise Preferences: In some settings, stakeholder may be able to rank 
alternatives for each criterion and the criteria themselves on their impor¬ 
tance, but may not want to commit on the strength of such preferences. 
Thus, evaluation of alternatives on a quantitative scale and weight esti¬ 
mation for criteria in QnMCDMs may involve assumptions extraneous to 
user input [S3] regarding the extent or strength of preference, questioning 
the validity of the results. 

4. Complex Tradeoffs: Stakeholders may tradeoff one set of criteria against 
another (specify a partial order over sets of criteria). For instance, ‘Im¬ 
prove on both efficiency and cost, if possible, even if it increases pollution,’ 
i.e., tradeoff Z against both X and Y (but not just one). QnMCDMs are 
clearly inapplicable in such scenarios, as any weight assignment will violate 
such a preference. 

5. Possibility of manipulation: Rankings produced by QnMCDMs are sen¬ 
sitive to changes in the attribute weights, e.g., rank reversals are known 
to occur even when the attribute weights are kept constant |14] . This 
raises the question as to which set of weights is faithful to the preferences 
of the stakeholder, and makes the results of QnMCDMs vulnerable to 
manipulation. 

6. Lack of Transparency: Stakeholders are often unaware of the mathemati¬ 
cal assumptions used in QnMCDMs and their implications on the decisions 
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m- The aggregation of criteria based on numeric weights makes it diffi¬ 
cult for stakeholders to understand the rationale behind a decision directly 
in terms of the stated preferences and rules based on principles of rational 
choice. 

7. Handling Component Dependencies: When each alternative is composed 
of multiple components, the preference over alternatives is a function of 
the preference over the components that make up the alternatives. Two 
components may make the alternative highly rated with respect to two 
different criteria, however when part of the same design, they may cancel 
out their contributions. For example, a certain choice of wall and paint for 
a building may be independently preferred, but the particular combination 
may release toxic gases raising safety concerns. QnMCDMs do not provide 
a way of accounting for component dependencies in such problems. 

3 Qualitative Preference Reasoning based Deci¬ 
sion Method 

We now describe a recently developed qualitative decision making procedure 
that relies purely on the qualitative relative importance preferences specified 
by the decision maker and uses mathematical logic to compute dominance and 
obtain an optimal ordering of candidates. This procedure was first introduced 
and studied in imiis], and has also been applied to several application domains 
such as pavement design in civil engineering |36j , requirements engineering |25] , 
sustainable design , and materials science and engineering [SD] . We now de¬ 
scribe how the qualitative decision method computes dominance by formulating 
and solving logical equations in contrast to existing decision methods, and hence 
produces an ordering of alternatives based on principles of rational choice. 

3.1 Qualitative Dominance 

Let A and B be two candidates described by a set A = {Xi, X2, ■ ■ ■ X^} of 
properties or attributes. Suppose that >-i is the intra-variable preference rela¬ 
tion on the domain oi Xi € X. For example, if Xi has its domain as the set of all 
real numbers then is often the natural total order on the set of real numbers. 
Further, let the binary relation l> represent the partial trade offs over attributes 
(relative importance among properties), e.g., Cost l> Performance means that 
Cost is more important than Performance. We say that A dominates B (equiv¬ 
alently, A is preferred to, or is better than B), denoted A>-d B whenever the 
following conditions hold: 

1. There exists at least one variable, called the witness X^ G fb, with respect 
to which A is preferred to B, i.e., A(Xyj) B(X^) 

2. A is better than or equal to B with respect to all attributes except those 
that are less important than X^ 
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If both the above conditions hold, then A is said to be preferred to B with 
respect to the relative importance tradeoffs of the stakeholder, i.e., A yd B. The 
above conditions can be succinctly expressed as an equation in mathematical 
logic notation: 

Ayd B 3Xu, G X : A{Xu,) y^u B(Xyj) A VX^ € X : X^j => 

A(Xk) tk B(Xk) 

The semantics or meaning of the above logical equation is as follows: O 
states that the left hand side (A y B) holds true if and only if the logical formula 
in the right hand side holds true. The right hand side of the above equation 
states the existence of a witness property G X : A(Au,) y^ B{Xu,)) such 

that A is not worse than B for all attributes that are not less important than 

A„. 

The advantage of the above method is that it can handle imprecision in 
attribute values for alternatives. Hence, it is possible to order alternatives that 
specify attributes in terms of interval range (as opposed to precise values), 
or where they are estimated on a qualitative scale (very expensive, expensive, 
inexpensive, etc.). 

3.2 Properties of the Dominance Relation 

In order to apply any qualitative decision method to compare and choose among 
alternatives, it must be checked for conformance to well established principles of 
rational choice. It is expected that a good dominance relation would satisfy the 
properties of irreflexivity, transitivity and asymmetry, which makes it a strict 
partial order. We formally state the definitions of these properties. 

Definition 1. Properties of Strict Partial Order Relation \73lj . A dominance 
relation >- is: 

1. irreflexive if for every alternative A, Ay A never holds 

2. transitive if for all triples A, B and C of alternatives, if A >- B and B y C 
both hold, then A y C 

3. asymmetric if Ay B then B )/- A 

Definition 2. Properties of Interval Order Relation HU. A relative impor¬ 
tance relation \> Q X x X is an interval order if: 

1. \> is a partial order, and 

2. for all Xi, Xj, A^, A; G A, A^O A^ and A^ > A; implies that either A^O A; 
or Afc > Xj 

It can easily be shown that an irreflexive and transitive relation is always 
asymmetric, and hence it suffices to show only that A is irreflexive and transitive 
to show that >- is a strict partial order. We recall the propositions from [29], 
relating to the properties of yd and state below the main result that qualifies 
yd as a rational dominance relation. 
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Theorem 1. When the input intra-variable preferences >-i for each Xi G X is 
strictly partially ordered and the relative importance (trade-offs) over the prop¬ 
erties > is interval ordered, the dominance relation >-d defined in Section [ffl\ is 
a strict partial order. 

3.3 Computational Complexity of Qualitative Decision Mak¬ 
ing 

The above described qualitative decision procedure involves more computational 
complexity to compute dominance and to rank the alternatives, in comparison 
with the 2-objective optimization methods or the MCDA methods like MAUT 
or AHP. In particular, the computational complexity for evaluating the logical 
equation in Section ld-ll for each pair of alternatives is of the order of mfi where m 
is the number of properties or attributes. Hence, if there are n alternatives and 
m properties from which the best alternative has to be selected with respect 
to tradeoffs, then it would require the order of computations to order 

alternatives, which is expensive (although possible with increased computational 
power). Despite the complexity, it may be worth using this qualitative method 
for several reasons. For example, in strategic decisions, one may need precision 
when incorrect ordering of two alternatives can have a large adverse impact to 
the decision maker. Another scenario that may need the rigor and precision of 
qualitative decision method is safety critical decisions such as site selection for 
harmful chemicals disposal. 

4 Conclusion 

There are several decision situations where the decision maker needs to order a 
set of alternatives from the best to the worst, purely based on possibly incom¬ 
plete, qualitative attributes describing the alternatives, and possibly incomplete 
qualitative trade-offs among the attributes. We have described a case for us¬ 
ing a qualitative decision method in such settings that provides a strict partial 
order of the alternatives as opposed to linear rankings provided by traditional 
quantitative decision methods. In the future we will examine the appropriate¬ 
ness of other advances in qualitative decision methods gi [Ml [Ml muni EH for 
different applications in engineering [TTl [Ml IMl [55l [59l EH EH , optimization- 
based methods [Ml EH EH [Ml EH , cybersecurity [MlEHEOl^ decision-support 
systems in healthcare [Ml EH EH EH EZl EH EH EH EH EH i ranking and rec¬ 
ommendation systems [Ml EH EH EH EH EH, service and cloud computing 
[H EH EH EH EH EH EH EH] , financial decision making [M] , and other areas of 
computing [6Z1IM1EHEHIII]- 
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